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Look  and  Feel: 

Haptic  Interaction  for  Biomedicine 

1995  Annual  Report 


1  Introduction 

People  use  their  hands  to  touch,  manipulate,  and  learn  about  the  world  around  them.  Interac¬ 
tion  through  touch  and  manipulation  is  known  as  haptic  interaction.  Rather  than  study  how 
humans  achieve  their  remarkable  skill  at  haptic  interaction,  we  are  working  to  build  machines 
that  amplify,  enhance,  and  transport  these  human  skills.  We  are  working  to  develop  advanced 
haptic  technology  that  allows  human  users  to  touch,  feel,  grasp,  and  manipulate  a  set  of  special 
objects:  objects  located  remotely,  objects  too  small,  too  large,  or  too  dangerous  for  normal 
human  interaction,  or  objects  that  exist  only  in  simulated  worlds  (virtual  objects). 

Advanced  haptic  interaction  will  have  a  wide  range  of  applications  in  biomedical,  military, 
and  civilian  areas.  It  will  allow  design  engineers  to  “pick  up”  new  designs,  “try  them  out,” 
and  see  how  they  wiU  behave,  before  they  ever  leave  the  drawing  board.  Engineers  will  sculpt 
the  shapes  of  new  parts  in  the  computer,  as  though  made  of  clay.  Remote  experts  wiU  aid 
the  local  novice  in  performing  delicate  procedures.  Medical  students  will  train  by  manipulating 
virtual  organs,  without  risk  to  human  patients  or  animals.  Students  will  probe  any  part  of 
the  virtual  patient  with  impunity  and  view  a  wide  range  of  conditions  from  any  vantage  point. 
This  work  will  enable  applications  ranging  from  familiarization  and  training  in  domains  requiring 
hand/eye  skills  (surgery,  mechanics,  maintenance,  for  example)  to  the  enhancement  of  operations 
in  remote,  scaled,  unfamiliar,  and  hazardous  environments. 

Systems  for  computer  haptic  interaction  need  to  have  three  parts:  a  part  that  mechanically 
interfaces  to  the  user’s  hand  (and/or  arm),  a  part  that  mediates  the  interaction  between  the 
hand  and  virtual  or  remotely  located  objects,  and  a  part  that  simulates  how  the  objects  will 
behave  when  touched.  These  components  are  force  feedback  devices,  haptic  algorithms,  and 
physics-based  simulation,  respectively.  In  support  of  the  surgical  simulator  and  other  related 
projects  we  are  building  a  series  of  advanced  haptic  systems  that  consist  of  a  mechanical  haptic 
interface  device,  a  real-time  computing  system,  a  dynamic  simulation  system,  and  computer 
graphics.  The  systems  we  build  provide  an  entirely  new  way  for  people  to  interact  with  remote 
and  virtual  objects  (Fig.  1).  This  report  describes  our  progress  in  developing  advanced  haptic 
systems.  It  describes  basic  technology  and  methods  we  have  developed  as  well  as  the  application 
of  the  technology  to  biomedical  haptics. 

The  immediate  goal  of  our  effort  in  biomedical  haptics  is  the  development  of  a  virtual  surgery 
simulator  that  will  allow  the  user  to  practice  an  anastomosis  (Fig.  2),  a  common  surgical 
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Figure  1:  Haptic,  Graphic,  and  Audio  Presentation  of  a  Virtual  Object 


procedure  that  involves  suturing  two  tubes  together  using  standard  surgical  tools.  A  long  term 
goal  of  this  effort  is  to  provide  a  setting  in  which  medical  students  and  surgeons  can  practice  a 
variety  of  surgical  procedures  under  conditions  that  are  better  for  learning  than  a  live  operating 
room.  Progress  in  surgical  simulation  will  also  contribute  to  the  field  of  teleoperation. 

2  Progress  on  Components  of  an  Advanced  Haptic  System 

The  primary  components  of  any  advanced  haptic  system  will  include  high  fidelity  haptic  inter¬ 
faces,  haptic  algorithms  and  dynamic  simulation.  In  Table  1  we  summarize  our  progress  on  a 
list  of  development  tasks  for  advanced  haptic  systems  including  the  virtual  surgical  simulator. 
Next  we  detail  our  progress  in  these  areas. 

2.1  Advanced  Haptic  Interface  Devices 

The  Phantom  (Fig.  3)  haptic  device  is  a  light  weight,  high  bandwidth  mechanical  interface  that 
allows  the  user  to  touch  and  manipulate  a  virtual  or  remotely  located  object.  Haptic  fidelity  and 
high  resolution  are  important  for  conveying  a  detailed  sense  of  touch  and  are  thus  emphasized 
over  kinematic  complexity  and  high  force  capacity.  High  fidelity  and  resolution  are  obtained 
by  maximizing  bandwidth  through  the  design  of  stiff,  lightweight,  and  low  friction  structures 


•  Advanced  Haptic  Interface  Devices:  The  haptic  device  is  the  mechanical  interface  that  allows  the  user  to 
touch  and  manipulate  a  virtual  or  remotely  located  object. 


—  Phantom  haptic  device  interfaced  to  SGI  and  PC 

—  Instrumented  medical  tool  attachments 

*  debakey  forceps 

*  needle  holder 

—  Virtual  medical  tools 

*  debakey  forceps 

*  needle  and  needle  holder 

*  scalpel  with  scalpel  holder 

*  hemostatic  forceps 

*  intestinal  forceps 

*  dissecting  forceps 

*  scissors 

*  retractor 

—  Instrumented  gimbal  measures  tool  position  and  orientation 

~  Powered,  two  finger  attachment  designed  and  tested  (MIT  Task) 

•  Haptic  Algorithms:  The  haptic  algorithms  mediate  the  forces  the  user  feels  with  the  Haptic  Interface 
Device. 

“  Two  handed  interaction  demonstrated 

—  Two  finger  manipulation  demonstrated 

—  Rigid  body  contact  force  models 

—  Flexible  body  contact  force  models 

—  Coulomb  friction 

—  Haptic  textures 

•  Physics-Based  Simulation:  Real-time  physics-based  simulation  is  used  to  calculate  the  behavior  of  objects, 
interactions  among  objects,  and  interactions  between  objects  and  the  user 

“  Automatic  simulation  generation  for  rigid  body  systems 
“  Loadable  object  shapes  from  CAD  files 
“  Programmable  bulk  properties  of  simulated  objects 

—  Compliant  flexible  skin 

—  Compliant  flexible  tubes 

—  Contact  detection  between  polygonal  shapes 

—  Contact  detection  between  point  and  flexible  tubes 

—  Contact  detection  between  point  and  terrain 

—  Surgical  needle  interaction  with  flexible  tissue 

—  Texture  mapped  graphics 


Table  1:  Progress  List  for  Advanced  Haptic  System  Development 
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(c)  ISfOC?  Boston  Dynamics  Inc. 


Figure  2:  The  Virtual  Anastomosis  Simulator:  an  artists  conception  and  a  working  prototype  at  BDI. 

and  transmissions.  The  Phantom  is  interfaced  to  both  a  PC  and  an  SGI  graphics  workstation. 
Haptic  algorithms  and  object  simulation  code  are  run  on  the  PC  while  the  graphics  workstation 
is  used  for  3D  visual  display. 

The  position  of  the  tip  of  the  Phantom  is  measured  using  encoders.  The  phantom  can  apply 
forces  back  to  the  user  along  the  three  translational  axes.  The  three  gimbal  option  measures 
the  orientation  of  a  probe  or  tool  attached  to  the  Phantom.  Attaching  instrumented  medical 
hardware  to  the  haptic  device  increases  the  realism  of  interaction  by  providing  a  familiar  interface 
to  the  user.  A  powered,  two  finger  device  has  been  designed  by  the  MIT  group  and  tested  at 
BDI.  This  device  can  apply  a  gripping  torque  to  the  user  to  simulate  the  sensation  of  grasping 
an  object  with  two  fingers.  A  similar  concept  could  simulate  the  forces  required  to  grasp  an 
object  with  a  forceps  or  pliers. 

Further  development  of  the  haptic  device  is  underway  and  will  include  piezo- vibration  stim¬ 
ulators  for  ultra-high  bandwidth  sensation,  multi-fingertip  interaction,  and  large  workspace. 

2.2  Haptic  Algorithms 

The  haptic  algorithms  mediate  the  force  the  user  feels  with  the  Haptic  Interface  Device.  These 
forces  may  be  the  contact  force  produced  by  pushing  or  pulling  a  simulated  object,  or  those 
from  poking  or  plucking  a  flexible  membrane,  or  they  may  be  the  forces  that  produce  a  sense 
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Figure  3:  The  Phantom  Haptic  Interface  Device 


of  textured  surface.  The  contact  force  models  we  use  incorporate  the  material  properties  of  the 
simulated  objects  and  are  dependent  upon  the  simulated  object’s  behavior.  Texture  forces  are 
produced  by  spatially  and  temporally  varying  the  contact  forces. 

2.2.1  Contact  Force  Models 

The  interaction  force  that  is  simulated  when  two  objects  touch  is  computed  with  a  contact  force 
model.  The  contact  force  is  computed  using  models  of  surface  compliance  and  friction.  The 
contact  force  is  also  dependent  upon  the  state  of  the  objects  in  a  simulation.  For  example,  the 
forces  required  to  touch  a  needle  to  flexible  tissue  depends  upon  whether  or  not  the  needle  has 
initially  penetrated  the  tissue.  These  types  of  contact  forces  depend  closely  upon  the  simulated 
behavior  of  the  object.  While  these  force  models  are  important  in  lending  a  convincing  sense  of 
touch  to  an  object’s  surface,  we  are  developing  other  techniques  to  add  richness  and  complexity 
to  haptic  interaction. 

2.2.2  Haptic  Texture  Mapping 

The  basic  goal  of  haptic  texture  mapping  is  to  permit  haptically  perceivable  textures  to  be  over¬ 
laid  on  basic  object  geometry.  We  have  experimented  with  creating  haptic  textures  by  making 
the  contact  forces  depend  upon  the  spatial  location  of  the  contact  point.  By  regularly  varying 
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the  contact  forces  with  spatial  position  we  have  created  the  feeling  of  ridges  on  a  geometrically 
flat  plane.  Using  this  idea  we  have  also  created  a  haptic  texture  from  a  scanned  in  photograph 
by  spatially  correlating  the  contact  forces  with  the  visual  image.  So  far  the  haptic  textures  do 
not  work  well.  The  best  effect  is  achieved  with  a  high  frequency  texture  but  we  do  not  yet  know 
how  to  generate  the  feel  of  real  textured  objects. 

2.3  Physics-Based  Simulation 

Our  system  uses  real-time  physics-based  simulation  to  calculate  the  behavior  of  objects,  inter¬ 
actions  among  objects,  and  interactions  between  objects  and  the  user.  Objects  can  be  isolated 
bodies,  mechanical  linkages,  or  more  complicated  things  like  tool  mechanisms,  deformable  tis¬ 
sues,  or  organs.  A  basic  idea  of  the  work  is  that  physics-based  simulation  allows  virtual  objects 
to  move  as  real  objects  do;  obeying  the  laws  of  physics,  they  feel  like  real  objects  when  touched, 
and  move  realistically  in  response  to  touching. 

The  foundation  of  our  object  simulator  is  software  that  we  developed  in  previous  work  that 
automatically  creates  highly- optimized  dynamic  simulations  of  things  that  move  from  simple 
descriptions  of  the  objects.  The  physical  simulations  include  material  properties  than  can  be 
changed  by  the  user  to  tune  the  feel  of  the  object.  For  example,  we  can  change  the  mass  of  a 
rigid  body,  the  compliance  of  its  surface,  or  the  coefficient  of  friction  between  a  surface  and  the 
haptic  device. 

2.3.1  Rigid  Body  Systems 

A  general  object  loader  is  working  that  allows  multiple  3D  objects  to  be  loaded,  dynamicaJly 
simulated,  and  touched  or  manipulated.  The  software  includes  automatic  contact  detection, 
equation  generation,  and  3D  graphics.  The  shape  of  these  objects  can  be  defined  with  a  CAD 
file  that  describes  their  surface  geometry.  The  geometry  is  used  for  both  visualization  and  contact 
detection.  Bulk  properties  of  the  bodies  like  mass,  coefficient  of  friction,  or  surface  impedance 
can  be  specified  by  the  user.  The  haptic  tool  used  for  manipulation  in  the  virtual  environment 
is  a  special  kind  of  simulated  object.  Its  position  and  orientation  are  driven  by  the  Phantom. 
The  forces  of  interaction  between  it  and  other  simulated  objects  are  computed  from  the  haptic 
algorithms. 

2.3.2  Flexible  Tissue 

To  support  the  simulation  of  surgical  procedures,  we  have  expanded  our  simulation  library 
to  include  prototype  simulations  of  flexible  tissue  and  flexible  tubes.  At  BDI  we  are  using 
flexible  tissue  simulation  techniques  that  strive  for  computational  simplicity  and  speed  while 
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providing  convincing  behavior.  One  prototype  flexible  tissue  simulation  included  a  planar  surface 
that  deformed  when  pushed  or  pulled.  Rigid  objects  with  varying  surface  compliance  could  be 
positioned  beneath  the  flexible  tissue  to  simulate  a  palpation  task.  We  also  have  simulations  of 
whole  and  severed  hollow  tubes.  The  tubes  move  when  touched  and  the  surfaces  deform  when 
the  tubes  are  poked  or  plucked.  By  varying  the  radii  of  the  tubes  with  time  we  can  simulate  the 
look  and  feel  of  a  pulsing  vessel.  The  flexibility  or  surface  deformation  of  a  flexible  tube  can  be 
tuned  to  best  approximate  the  feel  of  biological  material. 

2.3.3  Contact  Detection  and  Contact  Force  Models 

An  important  component  of  object  simulation  is  the  contact  detection  algorithm  that  determines 
when  simulated  objects  are  colliding  or  when  the  user  is  touching  a  simulated  object.  The  algo¬ 
rithms  use  the  geometry  of  the  simulated  objects  to  compute  the  distance  separating  them.  We 
have  created  general  contact  detection  algorithms  for  computing  the  distance  between  polygonal 
objects.  This  general  contact  algorithm  is  insensitive  to  the  number  of  polygons  on  an  object. 
We  also  have  written  special  case  algorithms  that  are  efficient  for  certain  applications  such  as 
contact  against  flexible  tubes  or  terrain  like  features. 

3  Haptic  System  Demonstrations 

We  are  compiling  a  suite  of  haptic  demonstrations  for  the  purpose  of  testing,  communicating, 
and  delivering  advanced  haptic  systems.  The  collection  of  demonstrations  provides  a  measure 
of  what  these  systems  can  do  in  existing  implementations.  The  demonstrations  rely  on  one  or 
two  Phantom  haptic  devices  that  are  interfaced  to  a  high  performance  PC  that  runs  dynamic 
simulation  and  haptic  algorithms.  The  PC  is  networked  to  an  SGI  graphics  computer  for  3D 
graphic  displays.  The  CrystalEyes  stereographic  and  head  tracking  product  has  been  included 
on  some  demonstrations  for  enhanced  3D  visualization.  Here  is  a  list  of  the  demonstrations 
currently  in  operation. 

•  The  Friction  House:  This  concave  piece  built  from  convex  shapes  is  fixed  in  space.  Using 
the  phantom  you  can  feel  the  shape  and  judge  the  friction  of  its  surface  as  you  drag  your 
finger  across  the  surface.  Forcing  the  graphic  image  of  the  virtual  finger  to  lie  on  the  surface 
of  the  house  during  contact  improves  the  sensation  of  hardness. 
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•  Toy  Box:  The  rigid  body  is  the  starting  point  for  3D  dynamic  simulation.  We  can  simulate 
the  interaction  between  static  or  dynamic  rigid  bodies.  The  body  shape  can  be  loaded  into 
the  system  in  a  standard  CAD  format.  Convex  rigid  bodies  or  concave  bodies  that  are 
comprised  of  convex  subshapes  can  be  touched,  felt,  and  manipulated,  bounced  off  the  floor 
or  walls  or  each  other.  We  have  built  a  set  of  these  objects  (a  jack,  a  block,  a  barrel,  and 
a  ball)  and  a  box  with  floors  and  walls  to  contain  them  while  you  poke  them  around.  We 
call  it  the  toybox. 

•  Flexible  Skin:  Using  forceps  the  user  can 
poke,  stroke,  or  pluck  a  sheet  of  skin. 

The  skin  is  compliant  and  deforms  when 
touched.  With  a  delicate  touch  the  user 
can  just  detect  lumps  under  the  skin. 

With  needle  and  needle  holder,  the  user 
can  practice  suturing  technique.  At  first 
the  skin  resists  the  needle  then  with  just 
enough  force  the  needle  punctures  the  skin. 

•  Flexible  Tube:  Roll  up  the  flexible  tissue  into  a  tube  and  you  have  a  blood  vessel,  small 
intestine,  colon,  or  one  of  many  biological  tubes  found  in  the  human  body.  The  tube  bends 
away  from  you  when  you  poke  it  or  toward  you  when  you  pluck  it  with  the  forceps.  This 
gross  deformation  of  the  tube  is  accompanied  by  local  surface  deformation.  The  tube  is  a 
bit  slippery  so  using  two  hands  is  recommended.  Hold  the  tube  with  the  debakey  forceps 
in  the  left  hand  while  puncturing  it  with  a  needle  and  needle  holder  held  in  the  right  hand. 
Release  the  needle  in  the  tube  then  reach  around  the  inside  and  pull  it  through  to  complete 
the  suture.  We  have  needles  pulling  thread  and  wiU  soon  have  the  virtual  anastomosis,  or 
the  tying  of  two  tubes  together. 

•  Haptic  Textures:  We  have  made  a  first  pass  at  displaying  haptic  textures.  We  created  a 
haptic  texture  by  spatially  varying  the  contact  forces.  The  best  effect  is  achieved  with  a 
high  frequency  texture  but  we  do  not  yet  know  how  to  generate  the  feel  of  real  textured 
objects. 

•  Hinged  Domino:  One  step  more  complicated  than  the  rigid  body,  the  hinged  domino 
features  two  rigid  bodies  hinged  together.  They  can  fold  closed  like  a  book  or  open  end-to- 
end  to  make  a  tower.  Friction  between  the  ground  and  the  dominoes  make  it  possible  to 
stand  the  tower  up  on  end  with  only  your  finger  tip.  Texture  maps  applied  to  the  dominoes 
improve  the  visual  realism.  A  dynamic  simulation  of  the  hinge  mechanical  structure  makes 
it  feel  like  a  real  articulated  object  when  touched. 
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•  Linkage:  Yet  more  dynamically  complex  is 
the  Vernier  Non-Linear  Nosewheel  Steer¬ 
ing  Linkage.  This  is  a  dynamic  simulation 
of  a  four  bar  linkage  used  in  the  steering 
mechanism  of  the  AV8B  Jet  Aircraft  (The 
Harrier).  Four  bar  linkages  can  have  a  wide 
variety  of  behavior  depending  on  their  con¬ 
figuration.  Manipulating  the  linkage  is  the 
easiest  way  to  begin  to  understand  its  be¬ 
havior.  Tighten  the  friction  nut  and  feel 
the  resistance  of  the  linkage  increase. 

•  Cockpit:  Static  objects,  one  degree  of  free¬ 
dom  dynamic  objects,  texture  maps  for  re¬ 
alistic  visuals,  and  sounds  recorded  from 
real  aircraft  trainers  make  up  the  sim¬ 
ulated  cockpit  of  the  AV8B.  By  push¬ 
ing  buttons,  turning  dials,  throwing  toggle 
switches,  and  reading  lighting  system  dis¬ 
plays,  the  user  plays  the  role  of  a  mainte¬ 
nance  technician  setting  up  and  diagnosing 
a  radar  system  test. 

•  Avionics  Bay:  A  continuation  of  the  radar 
test  started  in  the  cockpit.  Here  the  tech¬ 
nician  can  feel  avionics  components  in  a 
rear  bay  of  the  aircraft.  Using  a  voltage 
probe  the  user  can  measure  the  voltage  on 
an  electrical  connector.  Here,  feeling  the 
pins  is  crucial  to  keeping  the  voltage  probe 
in  place. 

4  Two  Handed  Anastomosis 


NoseWheel  Linkage 


Cockpit 


I 


Avionics  Bay 


As  a  means  of  pulling  together  the  basic  technology  required  for  biomedical  haptics,  we  are 
working  on  a  focused  application,  a  two  handed  anastomosis.  The  anastomosis  is  a  common 
surgical  procedure  that  involves  suturing  together  two  blood  vessels,  esophagus,  colon,  or  other 
biological  tubes.  The  dominant  features  of  the  anastomosis  that  our  simulator  currently  supports 
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are  the  use  of  common  surgical  tools  to  manipulate  and  grasp  compliant,  deformable  tubes, 
grasp  and  release  suturing  needles  and  insert  and  remove  suturing  needles  from  flexible  tubes. 
In  continuing  work  on  this  project  we  will  add  suture  material  to  the  needle  so  that  we  may 
begin  tying  two  tubes  together. 

We  have  constructed  a  Virtual  Surgery  Station  that  allows  the  surgeon  and  surgical  subject  to 
be  in  the  correct  relative  proximity  to  one  another.  The  user  stands  with  hands  at  table  height 
holding  a  debakey  forceps  in  the  left  hand  and  a  needle  holder  in  the  right  hand,  each  mounted 
to  a  Phantom  Device.  Looking  down  at  his  hands,  the  surgeon  instead  sees  a  mirror  reflecting 
the  computer  graphic  image  of  the  subject  and  the  virtual  tools.  This  image  is  produced  in 
approximately  the  correct  location  by  using  an  SGI  monitor  mounted  overhead.  A  hand  rest 
allows  the  surgeon  to  stabilize  his  hands  for  delicate  manipulation.  We  have  demonstrated  this 
Virtual  Surgery  Station  to  Surgeons  at  the  Pennsylvania  State’s  Hershey  Medical  Center.  They 
plan  to  incorporate  it  into  their  Virtual  Training  Hospital. 

5  Commercialization  and  Related  Work 

BDI  is  developing  and  selling  products  that  are  closely  related  to  the  ARPA  funded  work. 
Included  in  the  Appendix  of  this  report  are  several  one  page  fliers  describing  our  haptic  system 
products. 

•  BDI  is  marketing  Tangible  Reality(TM),  a  turn- key  virtual  environment  system  that  in¬ 
cludes  visual  images,  sound,  and  touch.  The  system  consists  of  a  force-feedback  device,  a 
real-time  simulation  computer,  a  physics-based  simulation  environment,  simulated  objects, 
real-time  3D  computer  graphics,  real-time  3D  sound  effects,  and  networking.  Early  versions 
of  the  product  will  support  standard  sets  of  physics-based  simulated  objects.  Later  versions 
will  support  user-defined  objects  and  deformable  objects.  This  product  provides  an  initial 
channel  for  commercialization  of  results  from  the  proposed  research.  For  example  “palpat¬ 
ing  in”  and  “haptic  texture  mapping”  are  obvious  candidates  for  enhancing  the  product  as 
we  understand  and  develop  them. 

•  We  are  selling  a  special  version  of  Tangible  Reality  designed  for  training  surgical  proce¬ 
dures.  The  first  version  of  the  system  wiU  support  two-handed  anastomosis  at  several 
scales.  Thomas  M.  Krummel,  M.D.,  Chair  of  the  Department  of  Surgery  at  Penn  State’s 
Hershey  Medical  Center  is  building  “The  Virtual  Hospital.”  The  VR  Anastomosis  Simula¬ 
tor  is  a  key  feature  in  his  concept  for  reducing  the  cost  and  improving  the  quality  of  medical 
teaching. 

•  We  are  working  with  Digital  Media  Associates  to  combine  haptic  displays  with  their  unique 
3D  display  system  for  medical  applications. 
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•  Musculo  Graphics.  We  have  developed  a  joint  project  with  Musculo  Graphics  Inc.  to 
develop  a  trainer  for  wound  debridement  for  the  U.S.  Army  at  Fort  Bragg.  Initial  phases 
of  this  project  involve  development  of  a  leg  and  leg  wound  simulator  with  haptic  tool 
interaction  for  training.  BDI  will  provide  the  haptic  components  of  this  project.  A  version 
of  this  simulator  is  included  in  the  FY97  P.O.M. 

•  Nissho  Electronics  Corp.  is  a  Japanese  distributor  of  the  BDI  Tangible  Reality  System. 

•  VR  Maintenance  Trainer  for  the  Joint  Advanced  Strike  Technology  Program.  This  was  a 
program  to  demonstrate  the  applicability  of  VR  technology  to  aircraft  maintenance  training. 
This  project  significantly  leveraged  our  ARPA-funded  project  and  offers  other  military 
applications. 

6  Conclusions 

One  year  into  this  project  we  have  demonstrated  the  feasibility  of  many  basic  components  of 
an  advanced  haptic  system.  We  built  a  baseline  haptic  system  for  simulating  and  interacting 
with  static  and  dynamic  rigid  bodies,  dynamic  multi-rigid  body  systems,  bodies  with  loadable 
CAD  shapes,  and  basic  deformable  tissues.  We  have  created  a  collection  of  demonstrations 
to  illustrate  the  working  components  of  an  advanced  haptic  system.  We  have  built  integrated 
haptic  demonstrations  for  aircraft  maintenance  training  and  a  surgical  anastomosis.  While  the 
technical  challenges  to  producing  a  complete  advanced  haptic  system  are  considerable, we  have 
not  yet  discovered  any  insurmountable  problems.  Never-the-less,  the  results  of  our  first  year’s 
effort  will  help  to  focus  our  effort  on  the  continuing  project  goals. 

The  complexity  of  our  demonstrations  are  frequently  limited  by  our  computational  resources. 
Our  contact  detection  software  is  under  continual  refinement  in  order  to  improve  efficiency  so  that 
we  may  experiment  with  ever  more  complicated  and  realistic  environments.  We  have  designed 
optimized  software  for  contact  detection  in  specialized  environments  such  as  those  containing 
convex  polygons  or  flexible  tubes.  We  need  to  expand  our  suite  of  fast  contact  detection  tech¬ 
niques  for  additional  specialized  environments  and/or  create  efficient  contact  detection  software 
for  more  general  environments.  Our  experiments  with  haptic  textures  were  not  completely  suc¬ 
cessful.  We  have  not  yet  been  able  to  produce  a  convincing  sense  of  touch  against  a  textured 
surface.  However,  our  effort  in  this  regard  has  been  limited.  We  have  ideas  we  have  not  tried 
and  will  invest  additional  energy  into  this  area  during  the  second  year.  Our  experiments  with 
multi-fingered  haptic  devices  have  been  limited.  What  experimentation  we  have  done  has  raised 
the  issue  of  the  need  to  provide  torque  feedback  to  the  user.  The  problem  is  not  just  one  of 
providing  a  convincing  sense  of  touch  to  the  user.  Torque  feedback  may  be  required  to  create  a 
well  behaved  interaction  with  a  physically  simulated  world  in  which  torques  must  exist.  Finally, 
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while  the  economic  cost  of  our  computational  platforms  is  not  excessive,  it  is  high  enough  that  it 
still  may  present  a  barrier  to  many  would  be  customers  of  haptic  systems.  In  our  effort  to  reduce 
the  cost  and  complexity  of  the  required  hardware  for  an  advanced  haptic  system,  we  wiU  consider 
porting  all  haptic,  simulation,  and  graphics  software  to  a  single  computational  platform.  Such 
a  system  may  have  limited  performance  on  existing  machines  but  could  provide  a  functional  set 
of  tools  for  many  would  be  customers. 


7  Appendix 

Following  are  three  one  page  fliers  describing  our  haptic  system  products. 


Tangible  Reality  ™ 


Tangible  Reality  Components: 

•  Haptic  Software  Library 

•  Force  Feedback  Device 

•  Haptic  Real-Time  Computer 

•  3D  Graphics  and  Sound 


BDI  Haptics  Software  Library. 

•  Kinematics  for  the  PHANToM 

•  Fast  contact  detection 

•  Contact  force  model 

•  3D  object  loader 

•  Dynamic  object  simulator 

•  Haptic  textures 

•  Deformable  tube  simulator 

•  Virtual  hand  tools 

•  User  interface 

•  Device  drivers  for  WindowsNT 

•  Network  communications 

•  Demonstration  software 


Tangible  Reality a  trademark  of  Boston  Dynamics  Inc. 
PHANToM™  is  a  trademark  of  Sens  Able  Devices,  Inc. 
WindowsNT‘S^  is  a  trademark  of  Microsoft  Corp. 


At  the  heart  of  the  Tangible  Reality  system  is 
BDI's  Haptic  Software  Library.  It  provides  the 
real-time  haptic  functions  needed  to  implement  an 
application.  The  Haptic  Software  Library  runs 
under  WindowsNT  on  an  Intel  Pentium  or  DEC- 
Alpha  computer  and  includes  3D  graphics 
software  that  runs  on  Silicon  Graphics 
workstations. 

The  force  feedback  device  is  the  PHANToM™,  a 
high-bandwidth,  low-ffiction  device  that  is  small 
enough  to  fit  on  a  desk-top.  The  real-time  haptic 
computer  reads  data  from  the  force  feedback 
device,  determines  contact  with  virtual  objects, 
simulates  object  behavior,  and  applies  forces  to 
the  user  through  the  force  feedback  device.  The 
haptics  computer  also  drives  real-time  3D 
graphics  displays  that  portray  the  visual  scene. 

BDI  uses  Tangible  Reality  systems  including  the 
Haptic  Software  Library  in  custom  applications 
including  surgical  simulation  and  aircraft 
maintenance  training.  Tangible  Reality  and  the 
Haptic  Software  Library  are  scheduled  for 
availability  in  Spring  1996. 

For  more  information  about  using  Tangible 
Reality  in  your  application  send  email  to 
info@bdi.com,  phone  us  at  (617)  621-2929  or 
visit  our  web  site,  http;//www.bdi.com 

About  BDI 

Boston  Dynamics  Inc.  (BD7)  creates  dynamic 
computer  simulations  of  things  that  move, 
including  humans,  animals,  robots,  and 
machinery.  Each  simulation  can  be  coupled  to  3D 
computer  graphics  and  haptic  feedback.  BDI 
customers  include  ARPA,  AT&T  Bell  Labs, 
DMA,  Pennsylvania  State’s  Hershey  Medical 
Center,  Mitsubishi  Electric,  NAWC-TSD,  Nippon 
Telephone  and  Telegraph,  Reebok,  United 
Technologies/Otis,  and  others.  These  customers 
have  applications  in  engineering,  training, 
biomechanics  and  entertainment. 
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VR  Aircraft  Maintenance  Trainer 

Goal:  Replace  several  physical  trainers  with  virtual  reality  trainers 


Boston  Dynamics  Inc.  (BDJ)  and  NAWC-TSD  are 
exploring  the  use  of  virtual  reality  technology  for 
JAST  aircraft  maintenance  training.  Can  the 
diverse  and  expensive  physical  trainers  currently 
used  for  0-level  maintenance  training  be  replaced 
with  virtual  trainers?  To  address  this  question  we 
modeled  two  training  maintenance  tasks  for  the 
AV8B  within  a  virtual  reality  maintenance  training 
environment,  A  videotape  demonstration  of  this 
project  is  available  from  BDL 


Benefits  of  Virtual  Reality  Trainers: 

•  Reduce  equipment  inventory 

•  Decrease  hardware  support 

•  Reduce  space  requirements 

•  Deployable 

•  Simplify  logistics  of  training 

•  Improve  configuration  control 

•  Realistic  training  at  reduced  cost 

•  Maintenance  training  occurs  prior  to 
availability  of  aircraft 

•  Evaluate  ergonomics  of  training 
procedures,  work  load,  etc. 

•  One  trainer  for  all  aircraft 

VR  System  Components: 

•  Physics-based  simulation 

•  3D  Graphics 

•  Interactive  haptic  device 

•  CAD  data 

•  Training  paradigm 


BDI 


VR  Anastomosis  Simulation 


•  Real-Time  Force  Feedback 

•  Physics-Based  Simulation 

•  Flexible  Parametric  Tubes 

•  Grasping  and  Needle  Penetration 

•  3D  Graphics  and  Sound 


A  surgeon’s  hands  have  a  remarkable  sense  of 
touch.  BDI’s  virtual  reality  surgical  simulator 
is  based  on  touch.  Force  feedback  devices  let 
the  user  feel  the  virtual  environment  in  addition 
to  seeing  and  hearing  it.  The  BDI  simulator 
focuses  on  open  surgical  anastomosis:  the  task 
of  suturing  tube-like  organs.  Anastomosis  is 
ubiquitous  in  the  operating  room,  and  includes 
blood  vessels,  esophagus,  ureters,  trachea,  the 
colon,  bile  ducts  and  other  organ  systems.  By 
integrating  high-fidelity  force  feedback  devices, 
surgical  tool  interfaces,  real-time  3D  graphics, 
and  physics-based  simulation,  BDI  is  providing 
an  effective  surgical  practice  space;  All  you 
need  are  the  scrubs  and  mask. 

The  BDI  surgical  simulator  is  built  on 
Tangible  Reality  ™,  a  system  consisting  of  force 
feedback  devices,  a  real-time  haptic  computer, 
BDI's  Haptic  Software  Library,  and  3D 
graphics  software.  Tangible  Reality  is  an 
integrated  system  that  allows  users  to  reach  into 
virtual  environments  with  their  hands  to  touch, 
feel,  grasp,  and  manipulate  simulated  objects. 


Boston  Dynamics  Inc. 

One  Kendall  Square 
Building  100 
Cambridge,  MA  021S9 

Telephone  617-621-2929 
Fax  617-621-1606 
Email:  info@bdi.com 


Reach  in  and  Touch  Something 


Tangible  Reality™ 


Tangible  Reality  Components: 

•  Haptic  Software  Library 

•  Force  Feedback  Device 

•  Haptic  Real-Time  Computer 

•  3D  Graphics  and  Sound 


BDI  Haptics  Software  Library. 

•  Kinematics  for  the  PHANToM 

•  Fast  contact  detection 

•  Contact  force  model 

•  3D  object  loader 

•  Dynamic  object  simulator 

•  Haptic  textures 

•  Deformable  tube  simulator 

•  Virtual  hand  tools 

•  User  interface 

•  Device  drivers  for  WindowsNT 

•  Network  communications 

•  Demonstration  software 


Virtual  Tools 


Tangible  Reality^**  is  a  trademark  of  Boston  Dynamics  Inc. 
PHANToM™  is  a  trademark  of  SensAble  Devices,  Inc. 
WindowsNT'^'^  is  a  trademark  of  Microsoft  Corp. 
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At  the  heart  of  the  Tangible  Reality  system  is 
BDI's  Haptic  Software  Library.  It  provides  the 
real-time  haptic  functions  needed  to  implement  an 
application.  The  Haptic  Software  Library  runs 
under  WindowsNT™  on  an  Intel  Pentium  or  DEC- 
Alpha  computer  and  includes  3D  graphics 
software  that  runs  on  Silicon  Graphics 
workstations. 

The  force  feedback  device  is  the  PHANToM™,  a 
high-bandwidth,  low-friction  device  that  is  small 
enough  to  fit  on  a  desk-top.  The  real-time  haptic 
computer  reads  data  from  the  force  feedback 
device,  determines  contact  with  virtual  objects, 
simulates  object  behavior,  and  applies  forces  to 
the  user  through  the  force  feedback  device.  The 
haptics  computer  also  drives  real-time  3D 
graphics  displays  that  portray  the  visual  scene. 

BDI  uses  Tangible  Reality  systems  including  the 
Haptic  Software  Library  in  custom  applications 
including  surgical  simulation  and  aircraft 
maintenance  training.  Tangible  Reality  and  the 
Haptic  Software  Library  are  scheduled  for 
availability  in  Spring  1996. 

For  more  information  about  BDI's  Virtual 
Reality  Anastomosis  Simulator  or  about  using 
Tangible  Reality  in  your  application  send  email  to 
info@bdi.com,  phone  us  at  (617)  621-2929  or 
visit  our  web  site,  http://www.bdi.com 

About  BDI 

Boston  Dynamics  Inc.  (BDI)  creates  dynamic 
computer  simulations  of  things  that  move, 
including  humans,  animals,  robots,  and 
machinery.  Each  simulation  can  be  coupled  to  3D 
computer  graphics  and  haptic  feedback.  BDI 
customers  include  ARP  A,  AT&T  Bell  Labs, 
DMA,  Pennsylvania  State’s  Hershey  Medical 
Center,  Mitsubishi  Electric,  NAWC-TSD,  Nippon 
Telephone  and  Telegraph,  Reebok,  United 
Technologies/Otis,  and  others.  These  customers 
have  applications  in  engineering,  training, 
biomechanics,  and  entertainment. 


BDI 


VR  Aircraft  Maintenance  Trainer 

Goal:  Replace  several  physical  trainers  with  virtual  reality  trainers 


Boston  Dynamics  Inc.  (BDI)  and  NAWC-TSD  are 
exploring  the  use  of  virtual  reality  technology  for 
JAST  aircraft  maintenance  training.  Can  the 
diverse  and  expensive  physical  trainers  currently 
used  for  0-level  maintenance  training  be  replaced 
with  virtual  trainers?  To  address  this  question  we 
modeled  two  training  maintenance  tasks  for  the 
AV8B  within  a  virtual  reality  maintenance  training 
environment.  A  videotape  demonstration  of  this 
project  is  available  from  BDI. 


Benefits  of  Virtual  Reality  Trainers: 

•  Reduce  equipment  inventory 

•  Decrease  hardware  support 

•  Reduce  space  requirements 

•  Deployable 

•  Simplify  logistics  of  training 

•  Improve  configuration  control 

•  Realistic  training  at  reduced  cost 

•  Maintenance  training  occurs  prior  to 
availability  of  aircraft 

•  Evaluate  ergonomics  of  training 
procedures,  work  load,  etc. 

•  One  trainer  for  aU  aircraft 

VR  System  Components: 

•  Physics-based  simulation 

•  3D  Graphics 

•  Interactive  haptic  device 

•  CAD  data 

•  Training  paradigm 


Tangible  Reality™ 


Tangible  Reality  Components: 

•  Haptic  Software  Library 

•  Force  Feedback  Device 

•  Haptic  Real-Time  Computer 

•  3D  Graphics  and  Sound 


BDJ  Haptics  Software  Library. 

•  Kinematics  for  the  PFIANToM 

•  Fast  contact  detection 

•  Contact  force  model 

•  3D  object  loader 

•  Dynamic  object  simulator 

•  Haptic  textures 

•  Deformable  tube  simulator 

•  Virtual  hand  tools 

•  User  interface 

•  Device  drivers  for  WindowsNT 

•  Network  communications 

•  Demonstration  software 


Tangible  Reality  ™\s  a  trademark  of  Boston  Dynamics  Inc. 
PHANToM™  is  a  trademark  of  SensAble  Devices,  Inc. 
WindowsNT™  is  a  trademark  of  Microsoft  Corp. 


At  the  heart  of  the  Tangible  Reality  system  is 
BDI's  Haptic  Software  Library.  It  provides  the 
real-time  haptic  functions  needed  to  implement  an 
application.  The  Haptic  Software  Library  runs 
under  WindowsNT  on  an  Intel  Pentium  or  DEC- 
Alpha  computer  and  includes  3D  graphics 
software  that  runs  on  Silicon  Graphics 
workstations. 

The  force  feedback  device  is  the  PHANToM''^^,  a 
high-bandwidth,  low-friction  device  that  is  small 
enough  to  fit  on  a  desk-top.  The  real-time  haptic 
computer  reads  data  from  the  force  feedback 
device,  determines  contact  with  virtual  objects, 
simulates  object  behavior,  and  applies  forces  to 
the  user  through  the  force  feedback  device.  The 
haptics  computer  also  drives  real-time  3D 
graphics  displays  that  portray  the  visual  scene. 

BDI  uses  Tangible  Reality  systems  including  the 
Haptic  Software  Library  in  custom  applications 
including  surgical  simulation  and  aircraft 
maintenance  training.  Tangible  Reality  and  the 
Haptic  Software  Library  are  scheduled  for 
availability  in  Spring  1996. 

For  more  information  about  using  Tangible 
Reality  in  your  application  send  email  to 
info@bdi.com,  phone  us  at  (617)  621-2929  or 
visit  our  web  site,  http://www.bdi.com 

About  BDI 

Boston  Dynamics  Inc.  (BDI)  creates  dynamic 
computer  simulations  of  things  that  move, 
including  humans,  animals,  robots,  and 
machinery.  Each  simulation  can  be  coupled  to  3D 
computer  graphics  and  haptic  feedback.  BDI 
customers  include  ARP  A,  AT&T  Bell  Labs, 
DMA,  Pennsylvania  State’s  Hershey  Medical 
Center,  Mitsubishi  Electric,  NAWC-TSD,  Nippon 
Telephone  and  Telegraph,  Reebok,  United 
Technologies/Otis,  and  others.  These  customers 
have  applications  in  engineering,  training, 
biomechanics  and  entertainment. 
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